Classification of Upper and Lower Face Action Units and Facial Expressions using Hybrid Tracking System and Probabilistic Neural Networks

The most of the human emotions are communicated by changes in one or two of discrete facial features. Theses changes are coded as Action Units (AUs). In this paper, we develop a lower and upper face AUs classification as well as six basic emotions classification system. We use an automatic hybrid tracking system, based on a novel two-step active contour tracking system for lower face and cross-correlation based tracking system for upper face to detect and track of Facial Feature Points (FFPs). Extracted FFPs are used to extract some geometric features to form a feature vector which is used to classify input image sequences into AUs and basic emotions, using Probabilistic Neural Networks (PNN) and a Rule-Based system. Experimental results show robust detection and tracking and reasonable classification where an average AUs recognition rate is 85.98% for lower face and 86.93% for upper face and average basic emotions recognition rate is 96.11%. Key-Words: Active contours, Action Units, Facial Expressions, Probabilistic Neural Networks